code-lists
Posted:
code-lists
Posted: | Source
Coding the Sikiliza interviews: the code lists
In this blog post we would like to tell you a bit more about the list of codes we designed to code our data, using the qualitative data analysis software program ATLAS.ti.
Code list design
In our previous blog we’ve shown you that ATLAS.ti helps you to analyze your data by enabling you to search for certain topics or themes in your data by retrieving quotations to which the codes representing those topics and themes have been applied. Ultimately, this means that the set of codes you decide to use, determines the kind and amount of information you will later be able to retrieve from your data for analysis, which is exactly why the code list is of essential importance to your research.
It is very tempting to design a very detailed and elaborate code list, in order to avoid the risk of “losing” any information that may later prove valuable for your research. The pitfall here is that this may cause your code list to become too large and messy, and therefore difficult to work with.
With our large data set and broad scope of research this was certainly an issue for us, but luckily we found that ATLAS.ti offers the functionality of doing complex data searches, i.a. on the basis of co-occurrences of different codes (within a quotation). Bearing this in mind, we have tried to design our code list in an efficient way, by eliminating codes without negating the options to retrieve very detailed information.
Because of the possibility to search for code co-occurrences it was, for example, sufficient to make plain codes for sectors (“education”, “water”, etc.) and plain codes for issues (“corruption”, “criminality”, etc.) for us to be able to compare the occurrence of issues within or between different sectors. This because we can simple re-use the sector codes in co-occurrence searches, instead of having to make sector specific codes for each separate issue, which would have come down to 5 codes per issue (e.g. “corruption within education sector”, “corruption within water sector”, etc.).
Building the code list...
In order to test Twaweza’s “Theory of change” - that argues that information will lead to public action and public action will lead to information - we have to look for relations between received information, public actions and reported results within the different sectors of education, health, water and sanitation.
As such, particular categories flow forth from Twaweza’s interests and purpose of research, which we have taken to serve as a framework for our code list. To deepen the scope of our analysis we have subdivided those categories into more detailed codes, based on topics and themes that we saw emerging from the data, whilst reading a sample of the interview reports.
Below is an overview of our different code categories (structured by function) with a short explanation about the codes they are comprised of:
Categories directly flowing forth from the pillars of Twaweza’s theory of change:
- Twaweza activity: Codes mentioning information or activities related to the Twaweza project.
- Information: Codes representing issues or problems in the villages.
- Commotion: Codes representing specific types of commotion by villagers such as anger, frustration, fear and being disillusioned.
- The concept of “commotion” (here meaning: The emotions people experience and the behavior they portray after receiving certain information and before actually undertaking action) is not part of the original “Theory of change”, but we decided to add this concept to our research topics, hoping that it will help us gain a better understanding of the ways in which specific types of information provision may or may not lead to specific types of community action. This could be valuable information for Twaweza in their mission to distribute messages that provoke the kind of reactions that will likely result in public action.
- Public action by villagers: Codes representing specific types of community actions, such as reporting issues to authorities, joining boycotts and cost-sharing to build facilities.
- Public action by authorities: Codes representing actions undertaken by the authorities to solve issues in the villages, such as providing resources, doing construction work or organizing meetings.
- Results: Codes indicating if public actions have had an effect on the situation in the village, either in a positive or negative way.
Categories for determining change and causality:
- Performance: Codes indicating if a sector is performing poorly, averagely or well.
- Change: Codes indicating if the overall situation within a sector has stayed the same, got worse or improved since the last interview.
Categories for cross-tabulation and co-occurrence searches:
- Type: Abstract codes which indicate if the information is about ‘the situation in the village’, ‘public actions’ or ‘results’.
- Sector: Abstract codes which indicate if the information is about the education-, water-, health-, sanitation- or other sector.
The first 6 code categories (Twaweza activity, Information, Commotion, Public action by villagers, Public action by authorities and Results) directly flow forth from the pillars of Twaweza’s “Theory of change”, and their codes are meant to give us a deeper insight into the manifestation of different types of issues, commotions, public actions and results, and the frequency in which they occur. The following 2 categories (Performance and Change) add the dimension of time to our evaluation. For instance, allowing us to look at (and/or compare) sector performance within specific time spans, or to evaluate information over the course of time and determine the (causal) nature of relations found between the codes of the previous 6 categories. Finally, the codes from the last 2 categories (Type and Sector) can be used on their own to explore the data on a very abstract level, or in combination with other codes, to narrow down your search for a topic to the scope of - for example - a single sector (e.g. the topic of “corruption” within the health sector only).
Codes, as well as documents, can also be aggregated into so called “code families” and “document families”, opening up new possibilities to cross-tabulate and explore topics and sources.
It is important to note that the code list is not a static, lifeless object, as it is changing all the time as we find new relevant topics or come to adjust our understanding of the importance of certain issues whilst looking through and coding the data. Through the ongoing process of adding new codes, removing redundant codes, or aggregating or diffusing codes that prove either too specific or abstract, the code list is evolving into something that best suits our research goals.
… and making it look good
Our full code list currently counts 227 codes, and is therefore too long to display here. We can however show you some snapshots of it:
2 snapshots from our current code list.
The only possibilities for organizing your code list that ATLAS.ti offers are arrangement by alphabetical order and application of color. If you look closely at the codes on the code list you see that all the codes are preceded by a prefix, which we use to sort together codes under their appropriate (sub)category. Furthermore, all sector specific codes have been assigned a sector specific color, not only giving the code list a clear and organized appearance, making it as easy as possible for everybody to retrieve the code they are looking for, but also making it look absolutely gorgeous!
Working with Atlas.ti
Posted:
Working with Atlas.ti
Posted: | Source
Hurray for technology!
The days of colored pencils, tedious quote clipping and filing cabinets are over. Rapid advancements in technologies for computer assisted qualitative data analysis are making it easier and easier to handle increasingly large data sets and efficiently explore data in more and more detail. Offering coding-, linking-, networking, memoing- and querying tools, these technologies open up new possibilities for qualitative data management and analysis, which would be incredibly time consuming or downright unthinkable if had to be done manually.
With the Sikiliza project we got the exciting, but also challenging, task to work with such a large data set (5145 interviews from March 2012 to September 2013, and counting…) making use of the software program ATLAS.ti.
In this blog post we would like to take you on a small journey into the wondrous word of 21st century qualitative data analysis, and give you a taste of what it is like for us working with this software.
How does it work? The first thing you need to know about computer assisted qualitative data analysis (CAQDA), is that it is made possible by the grace of “coding”. This means that all pieces of data (in our case: pieces of interview reports) need to be categorized and labeled on the basis of their contents, using one or more codes, which are representatives for certain topics or themes of interest to your research. The software enables you to later explore a certain topic or theme by retrieving all quotations to which a code, or a combination of codes, is applied.
Unfortunately, for all its benefits, the CAQDA technology is not quite at the level where it does ALL the work for you. So, disappointingly ;), it turned out that we had to get our hands a little dirty after all and had to start labeling all the interviews with a large number of codes representing all the topics and themes that we expected could possibly be relevant for our research.
* For more information about our code list, read our next blog post on this website.
Meet the software
Now, let us give you a glimpse of the ATLAS.ti software. This is what our project in ATLAS.ti looks like:
1. Our project in ATLAS.ti
2. Snapshot
In the left pane of picture 1 you see one of our data files (an interview report). In the middle pane you see the codes that have been applied to the quotations. The vertical lines in front of the codes mark the start- and end point of the quotation in the interview report. In the right pane you see the code list, from where the applicable codes are dragged onto the quotations.
If you take a better look at the interview report you see that it is organized into “snippets” (pieces of text) based on their type (situation in the village, public actions undertaken, or results) and the sector the information is about (education, health, water, sanitation or other). For all these different types and sectors we have created separate codes, which can be used in cross tabulations, allowing us to perform targeted searches for issues within a specific sector, or compare information between different sectors. The ‘attributes’ at the head of the snippets enable us to automatically code the full snippets with the appropriate type and sector codes, using ATLAS.ti’s auto-coding tool.
3. Snapshot of an interview converted into a txt. file for easier use in ATLAS.ti. The ‘attributes’ in the first snippet tell you this piece of the report is about the situation (Info) within the health sector (Health). The second snippet is about public actions that are or aren’t undertaken (PubAct) within the health sector (Health). The third snippet is about the situation (Info) within the water sector (Water), and so on.
The challenges of team work
Our research team consists of quite a lot of members, who all have to work in the same ATLAS.ti project. Team work always brings along some extra complications for the work flow, as strict arrangements have to be made about when things are done and how the research is to be carried out. With the use of this type of software a lot depends on the consistent use of codes and other functionalities within the program, making frequent fine tuning essential. Also, the way in which projects in ATLAS.ti are set up technically, makes the team work an even more delicate enterprise.
A project in ATLAS.ti consists of a document folder (in our case, all interview reports) and a so called “HU file”, which comprises the path references to all the documents that are assigned to the project and all the codings and changes made to these documents within the project.
When working in a team, every member gets his or her own HU file and document folder to work in and once every 2 weeks all the hard work of all the team members is merged together by our “project administrator”, giving birth to a new HU file which contains all the work done by all the team members. This HU file is then redistributed to all team members, after which they proceed their work in this file, and so on.
This may sound fairly straight forward, but especially in the beginning we encountered a lot of technological difficulties, installing, opening, managing and synchronizing the projects on all our different computers, sometimes resulting in people losing some of their work. Therefore, it didn’t take long before the officially called “HU” files, were lovingly rechristened the “Hate-You” files by our research team.
Luckily, we are getting better and better at detecting and solving these kinds of issues, leaving much more time for the thing it is all about: coding, exploring, and making new discoveries!
Exploring the data
In order to explore your data landscape, ATLAS.ti offers both quantitative tools that generate output in numbers and show you how frequently codes occur on their own or together with other codes; and qualitative tools that help you yield all the quotations that are labeled with a certain code or combination of codes. As such, the quantitative tools are best used to help you create a general overview of “what might be going on” on a broad level. The qualitative tools will help you to look deeper into certain issues by retrieving all relevant segments of text for you to read through and interpret.
To give you a better idea of the type of outputs we can create from our coded data, we have selected a couple of interesting examples to show to you. These examples are just a tip of the iceberg as our detailed code list allows us to search and explore the data in an endless amount of ways.
4. Code frequency table about problems in the education sector.
This table shows you the number of times the codes displayed on the vertical axis (all representing a specific problem in the education sector) have been used. You can read from the table that the most mentioned problems by our respondents are; poor performance on examinations (255 x coded); shortage of teachers (149 x coded), school buildings/class rooms (105 x coded) and desks (93 x coded); and students skipping school or dropping out (together 107 x coded but possibly with some overlap).
5. Co-occurrence table of community actions per sector
The table above is an example of how cross tabulation can be used to compare issues between different sectors. It is made by cross tabulating the “sector codes” against the “community action codes”, and shows you the number of times specific community actions (displayed on the vertical axis) are mentioned within the different sectors (displayed on the horizontal axis). Most community action seems by far to take place within the education sector (431 x coded), followed by the water sector (181 x) and the health sector (179 x). In comparison, not much community action seems to take place within the sanitation sector (only 67 x coded). The table also contains some interesting information about people’s preferences for specific types of community action. Most of the actions mentioned seem to be people solving their own issues by financing, constructing or otherwise solving things by themselves, rather than asking or demanding help from the authorities.
If you once more look at the table above, you may notice that the most frequently occurring community action is “villagers financing / cost sharing by themselves” (100 x coded). This yields the questions what exactly the villagers are financing and why they tend to do so. To answer these questions it is necessary to read through and interpret the contents of those quotations. This concluding example shows you the output as it is generated by ATLAS.ti’s quote retrieval mechanism, i.e. the query tool:
6. Output of the query tool: all the quotations that are labeled with the public action code “PA(V). Villagers financing or cost sharing..” AND the sector code “_Education”
Working with ATLAS.ti in 2014
After a full 6 months of coding we are now approaching the finalization of the coding process. In these past months we have experienced that the more data we code, the better we are able to discern important topics and plausible relations between them from the data, and thus, incidentally, the more interesting our work gets. We are therefore already looking forward to the final analysis, which we are of course going to perform with the help of our dear friend – and from time to time worst enemy – ATLAS.ti!
Coding Interview Reports
Posted:
Coding Interview Reports
Posted: | Source
Interview, part III: Coding the Interview Reports
This third and final part of the interview with our research team member is about his experiences working with Atlas.ti: The computer software we use to manage and analyze the massive amounts of qualitative data we gather through the telephone interviews. In Atlas.ti pieces of text can be categorized and labeled with so called “codes” so they can be retrieved later by the researcher, enabling him or her to quickly explore different topics and correlations between them.
Snapshot from the Atlas.ti software: a ‘coded’ interview.
Since half a year you have been coding in Atlas.ti. (software for qualitative data analysis) Can you tell us something about that?
It’s very interesting. We have a lot of data so it is good that we have Atlas.ti to help us with the management and the analysis. Coding is a very friendly way of understanding the data.
At the beginning working with Atlas.ti was difficult. For instance, I made my own codes for everything instead of using the codes from the shared code list. Another thing was that we encountered all kinds of technical problems, opening and installing the projects on our computers.
It was very helpful that Kimberley (our qualitative researcher from AIID Amsterdam) came to Tanzania last September, to explain more about the software and the coding. Before we were in darkness. Now we know each and every little bit, so at last we are getting somewhere.
At the beginning of the project it was difficult to imagine how everything would work. “We have a lot of data, but how are we going to do it? How are we going to analyze?” But currently I realize it, I feel it. It has become simple, different from the beginning.
Training in the use of the code list, September 2013 at IRDP, Dodoma.
When do you do your best coding?
I work on coding early in the morning, because for coding you need to be concentrated. I usually start coding around 4 am. After 5 am, I go to the office, and I’m already somewhere. In the office I do a little bit more coding. After the coding I spend 2 or 3 hours doing the telephone interviews with the respondents. After that I’m working on my dissertation proposal.
Some of our research team members diligently coding interviews in Atlas.ti, September 2013 at IRDP, Dodoma.
Have you tried to do any analysis yet?
Yes I’m always trying to do some simple analysis for my PhD research. For instance, exploring the issue of “lack of medicine in the health centre” by frequencies of word occurrences, etc. We are all trying to do simple analyses, and always checking what the Sikiliza data is telling us about the different objectives we have in our proposals. But, we need to go further with the analysis so we are hoping we will get the latest version of Atlas.ti which has more advanced tools for analysis than the version we are working with now.
What do you think the advantages of working with Atlas.ti are?
It allows you to summarize the whole project you are working on. I also like that it simplifies dealing with the details. For instance, if you want to understand the problem of students that are dropping out of school, you can get all the information you need very quickly by clicking on the code “students drop out”, which gives you all the quotations you are looking for. It helps us to grasp the real situation.
Interviewing Respondents
Posted:
Interview, part II: The Phone Interviews"
In this second part of the interview we ask our research team member about his and his colleagues’ experiences with conducting telephone interviews with their respondents from all over Tanzania.
Our research team member has been calling his respondents once or twice every month over the past 2 years to ask them about problems and happenings in their villages. This intensive and enduring contact between interviewer and interviewee yields some very special stories to tell.
Image(filename="/Users/Chris/Documents/Tanzania/blogs/b2p1.png",embed=True,width=200)
What are the things you like about doing interviews?
I like the fact that my respondents are very, very open with me. They feel free to tell me the truth. They share their pain or joy with me if something happens in their village.
Sometimes respondents ask us to solve a particular problem they have in their village. They seem to think we are able to solve all their problems. This can be a bit difficult because we are not in a position to help them out on these issues in a direct manner. So we try to explain to them that we are collecting and recording the information they give us in order to write reports which can be submitted to authorities and can be used for policy influence and policy making.
Are there things you dislike about doing the interviews?
Yes, sometimes you have to talk a lot, and with the introduction of the new interview protocol the interviews take up even more time. Depending on the issues in a village, a single telephone conversation can take over 40 minutes. This also has financial implications, particularly with the recently increased communication expense rates. Nevertheless, we are trying our best to manage.
Below a snapshot of the new interview protocol. In table one (above) a rough summary of the conversation is sketched by the interviewer. The interview is later written out in more detail in a second table, which is used for analysis.
Image(filename="/Users/Chris/Documents/Tanzania/blogs/b2p2.png",embed=True,width=500)
How do the respondents react when you call them?
Before, people often found our frequent calls a disturbance, because they felt they didn’t benefit from it. Luckily we managed to solve this problem at the regional meetings.
Currently, all respondents like it when I call them. Sometimes they even call me themselves to tell me about new problems or events. Imagine that!
You call your respondents once or twice every month. How would you describe your relationship with them?
We talk a lot with our respondents, and therefore it feels like we have become friends. With some of our respondents we are really close. We talk about our personal lives and sometimes these respondents even tell us their family secrets or ask us for help with personal issues. With my favorite respondents I joke and laugh a lot.
In the course of the Sikiliza project you have collected a lot of stories about the villages and the problems that they face. Which of these stories did you find the most memorable?
I remember one very sad story from Katindi village about corruption in health services. A husband went to the hospital with his pregnant wife who was sick. At the hospital the couple was expected to pay the health staff a bribe in order to receive services, but they didn’t have any money. As a result, the staff refused to give the woman treatment and both the woman and the child died in the hospital.
During the interviews you have also gathered a lot of information about the different kinds of actions that people undertake in order to improve the situations in their villagers. Which story about public action did you find the most touching or thrilling?
There are so many stories, so it is hard to pick one. One story is about a troubled husband who one day sent his wife from home and then tried to kill his firstborn son. The neighbors heard the boy cry and came to rescue him. They took him to the hospital for treatment but it was already too late and the boy died of his injuries. The issue was reported at the police station, but the boy’s father bribed the police, and nothing happened. But, the villagers were very angry and they demanded their right, forcing the government to act against this person. As a result, the boy’s father was called into the police station and was taken to court.
After all the interviews you have done, do you think that people – by undertaking public actions - can really manage to bring on improvements in their villages?
Definitely! It is true that public action, especially community action, can help people achieve a lot. For instance, in one of our villages, the people had decided to contribute all their money to build a dispensary. And now all families are contributing 17.000 Tanzanian shilling per month to build a laboratory for the secondary schools. You see similar things happening in other villages as well, for example in Mrusi Street (Kigoma) where the villagers recently built a doctors house with money they collected among themselves. So you see, community action can help a lot. If all the villagers of Tanzania would have this kind of spirit, we could get very far as a country.
The power of community action as shown in the “Twaweza ni sisi” short movies series (http://www.youtube.com/user/Twaweza)
Image(filename="/Users/Chris/Documents/Tanzania/blogs/b2p3.png",embed=True,width=500)
Do you think people find it difficult to actively demand help from the authorities?
I don’t think so. So many people are demanding to have dispensaries, schools, water facilities, etc., but the issue is that the government doesn’t respond to their demands, so the only solution is to do it yourself. Most of the projects you see happening are initiated, financed and carried out by the villagers themselves, because the government generally doesn’t respond positively and quickly enough to villagers’ requests. One of the other research team members is conducting interviews in Singida, where the people are facing water shortages and are demanding water infrastructure for their villages. The government keeps refusing to do anything about it, and now you see that the villagers have started to drill their own wells. People have to find their own solutions, otherwise the problems will remain.
Do you think villagers’ attempts to demand things from the authorities can be fruitful at all?
Yes, they can be successful. For instance, at Tabora, the people were demanding better water facilities. The government responded by installing water pipes. On the other hand, in Kayanga the people are actively demanding a dispensary, but to present nothing has been done there.
I think the chances of the authorities responding to villagers’ demands may be dependent on the area. If the political leaders of your village are from the opposition party, you can demand as much as you like, but nothing will happen. But if your village leaders are from the ruling party it is much more likely that the government will respond positively to your requests.
Sikiliza: Meet the Callers
Posted:
Sikiliza: Meet the Callers
Posted: | Source
Interview, part I: The Regional Meetings
It’s the start of 2014, which means that it has already been 2 years since we started collecting the stories of 250 villages from all over Tanzania, by conducting interviews using mobile phones.
A lot has happened in the past 2 years, and what better way to inform you about our experiences with this special type of research methodology then through an extensive interview with one of our own research team members, who are calling and interviewing respondents as a daily practice.
So, for your education and entertainment, we have turned our research team member inside out with questions. We hope you’ll enjoy his stories and opinions about interviewing respondents, organizing respondent meetings and working with computer software for qualitative data analysis.
The interview will be published in a series of three separate blog posts, all comprising a different topic.
In this first blog post we have asked our research team member about the regional meetings that were held in august 2013. At these meetings, all the respondents participating in the Sikiliza project were called together at a regional level for a discussion about the project’s proceedings. During these meetings the respondents received a gift for rewarding and ensuring their continuing participation in the project.
Here's a picture of a regional meeting held in Dar.
What was the reason the regional meetings were organized?
Before the regional meetings we had problems reaching some of the respondents. Sometimes we called people and they wouldn’t pick up their phones or they would cut off the call. It was very difficult, because these respondents felt we were disturbing them; calling and calling and calling them, without giving them anything in return. They felt like we were using their time and didn’t see how they were benefitting from the project. So, we organized the regional meetings to meet and talk with them and to surprise them with a new phone!
Who organized the meetings?
We (the research team members working on the listening device) organized the regional meetings ourselves. When I called my respondents and told them to meet me at the district centre for the meeting, and that we would pay for their travel costs and accommodation, they became very happy. Traveling from their village to the district centre was a privilege to them.
Did you encounter any problems whilst trying to organize the meetings?
Yes, we had some problems trying to reach all the respondents, because some of them live in very remote areas. We tried to solve this by sending those respondents a text message a couple of days in advance, informing them that we were coming over for the meeting and asking them to “beep my phone” as soon as they had network-access. After I’d received a beep, I’d call them back and arrange the appointment with them. The network is always a difficulty, it’s very problematic.
Another problem is that it took up a lot of our time to travel to all the different districts for the meetings. For instance, I had to travel from Dodoma to Kigoma and then from Kigoma to Sumbawanga, which took up to 3 days.
Can you describe the regional meetings’ proceedings?
At the regional meetings we met the respondents and talked about the challenges they are having. We asked them what they think about the program, and how they think it is going so far. We had drinks together and I gave them their new phones. During the meetings we also gave the respondents a questionnaire which they filled in and returned to us. I think we spent about 1 to 1,5 hour with them.
Here you can see respondents fill in the questionnaires
Can you tell me more about the benefits of the regional meetings?
I think after the regional meetings more respondents are happy. They are giving us feedback. For instance, I went to Shinyanga and found they responded very positively. Some respondents said: “At least now you are here and we can ask you some questions!” They want personal contact, you see. And this personal contact helped us a lot. We gathered a lot of information during the regional meetings, and currently we have good response (to calls) from them!
Before the regional meetings respondents were asking us for things, for example money or solar power and solar chargers, and were telling us: “Why are you calling us all the time and using our time, while we don’t gain anything from it?” You have to keep the respondents happy, otherwise you don’t get the information you need. This is a challenge though, because it is true that we are always calling and calling them, but after the regional meetings the respondents have become happy and this has changed. We have good contact with the respondents now. It really is different from the beginning.
The regional meeting even resulted in me finding new respondents for villages where we lost contact with the initial respondent, for example Saku in Shinyanga, or where we haven’t been able to find a respondent before. So, I am now up to 59 respondents!